Image pickup apparatus

ABSTRACT

An image pickup apparatus including: CCD image pickup device; a light blocking plate for shutting out light incident upon the image pickup device; an exposure period controlling section for setting exposure time by controlling the light blocking plate and the image pickup device; a dark signal storing section for storing dark signals obtained from the image pickup device in a state where light is shut out by the light blocking plate; a subtracting section for subtracting the dark signals from main image pickup signals of the image pickup device; a defect detecting section for detecting defects from the subtracted image pickup signals outputted from the subtracting section; and a defect correcting section for performing correction of the detected fault pixels. It is thereby possible to achieve an image pickup apparatus capable of detecting and correcting at high accuracy those fault pixels occurring afterwards because of changes due to the passage of time or those fault pixels transiently occurring under certain modes.

[0001] This application claims benefit of Japanese Application No.2001-154805 filed in Japan of May 24, 2001, the contents of which areincorporated this reference.

BACKGROUND OF THE INVENTION

[0002] The present invention relates to image pickup apparatus in whichpixel defects occurring on image are adaptively detected and correctedin accordance with an image scene or an operating state of theapparatus.

[0003] The rate of occurrence of defect pixels, i.e., fault pixels isgenerally higher in such image pickup apparatus as a high-definitionelectronic camera where a solid-state image pickup device having a largenumber of pixels is used.

[0004] Technology for the detection and correction thereof has becomeindispensable. The yield of solid-state image pickup device becomeshigher and the price of the apparatus is greatly reduced by correctingpixels of singularities represented by these fault pixels.

[0005] The known technology for electrically correcting such faultpixels includes the following techniques. In particular, Japanese patentlaid-open application 55-156482 for example discloses the technique ofpreparing a memory in which locations of fault pixels occurringpeculiarly to each solid-state image pickup device are previouslyretained at the time of manufacturing the solid-state image pickupdevice. Such a memory is mounted on the image pickup apparatus such asan image sensor. While continuously surveying output signals from thememory, the fault pixels at the predetermined locations are interpolatedfor example by means of average values of adjoining pixels.

[0006] Further Japanese patent laid-open applications Hei-6-6685 andHei-9-289614 for example disclose the technique of providing a mode todetect fault pixels under certain image pickup conditions for example atthe time of introducing power or at the time of an adjustment involvingthe operation for shutting out light incident upon the solid-state imagepickup device so that, though a memory is provided, information on thedetected defects is updated at each occurrence of such certain imagepickup conditions.

[0007] Further Japanese patent laid-open applications Hei-7-23297 andHei-9-247548 for example disclose the technique of making correction bydetermining an arbitrary pixel of an arbitrary image as to its defectbased on its correlation with the surrounding pixels duringphotographing operation of camera.

[0008] Furthermore Japanese patent laid-open application 2000-59690discloses an electronic camera which previously retains location data offault pixels in a memory and at the same time has a function capable ofdetecting defects also from an arbitrarily selected taken image inaccordance with the exposure time so that it can deal with transientdefects occurring at the time of a long exposure.

[0009] Now, the one disclosed in the above described Japanese patentlaid-open application 55-156482 requires an exclusive memory for eachindividual sensor, i.e., each image pickup apparatus such as a camera.It cannot be jointly used by separate image pickup apparatus. Further,it cannot deal with the types of defect occurring due to the passage oftime after the shipment of the image pickup apparatus. Furthermore,there is also a problem of increase in both price and dissipation powerdue to the fact that memory size for storing defect locations becomesvery large proportionally to the increase in the number of pixels as aresult of achieving a high-definition image. Moreover, there is anotherproblem that it cannot deal with the types of defects which occurtransiently in a long time exposure or under high temperatures.

[0010] The ones disclosed in Japanese patent laid-open applicationsHei-6-6685 and Hei-9-289614, on the other hand, are capable of dealingwith those defects which occur due to the passage of time. These,however, cannot deal with those transiently occurring defects such asthe defects in the case of a temperature hike after continuation ofnormal image pickup conditions, since the certain image pickupconditions for detecting defects are the conditions outside the normalphotographing. Moreover, in the case of detecting defects under certainimage pickup conditions, since it is necessary to enter the defectdetecting operation mode with limiting to such certain image pickupconditions, there is a problem that images cannot be taken during a longtime period of such certain defect detecting operation mode.

[0011] Further, the techniques disclosed in Japanese patent laid-openapplications Hei-7-23297 and Hei-9-247548 are capable of dealing withchanges after the shipment or in temperature, since detection/correctionof defects is performed for each image. There are many edge patterns,however, in an arbitrarily selected image and it is difficult to detectdefects around edge. An attempt for accurately detecting the defectstherefore results in a problem that the size of circuit and programbecomes very large. In addition, while a high processing speed is aprerequisite in detecting defects in real time for each taken frame, anexceedingly large circuit and program size slows down the processingspeed and makes a real time defect detection and correction thereofimpossible.

[0012] Furthermore, while the electronic camera disclosed in Japanesepatent laid-open application 2000-59690 is adapted to be capable ofdealing with transient defects occurring at the time of a long timeexposure, it is not uncommon that the occurrence of such transientdefects under a long time exposure is concentrated in a small area. Insuch a case, an accurate detection of the defects is difficult. Also,there is another problem that the technique disclosed therein isincapable of dealing with those defects resulting from changes due tothe passage of time after the shipment of the electronic camera.

SUMMARY OF THE INVENTION

[0013] To eliminate the above problems in image pickup apparatus havingthe known defect detecting/correcting functions, it is an object of thepresent invention to provide an image pickup apparatus in which pixeldefects occurring on image are detected and corrected adaptively to theimage scene or the operating state of the apparatus so that it ispossible to precisely correct those fault pixels occurring afterwardsbecause of changes due to the passage of time or those fault pixelstransiently occurring under certain modes.

[0014] In accordance with a first aspect of the invention, there isprovided an image pickup apparatus including: an image pickup device; alight blocking means for shutting out light incident upon the imagepickup device; a subtraction means for subtracting a dark signalobtained as an output signal of the image pickup device when shuttingout the incident light from a main image pickup signal obtained from theimage pickup device at the time of a main image taking; a correctionmeans for correcting defect signals occurring due to fault pixels of theimage pickup device with respect to the subtracted image pickup signalfrom the subtraction means; and a control means for controlling thelight blocking means consecutively before or after the main image takingso as to obtain a dark signal of a time duration corresponding to thenumber of occurrence of externally caused fault pixels of the imagepickup device. In accordance with a second aspect of the invention, thecontrol means of the image pickup apparatus according to the firstaspect controls the light blocking means in accordance with an exposuretime of the main image taking.

[0015] In image pickup apparatus, defects occur in every portion withinan image and the defect detecting accuracy is greatly reduced forexample at the time of a long time exposure. In the image pickupapparatus according to the above described first and second aspects,however, dark signals obtained by shutting out light for a time durationcorresponding to the amount of occurrence of fault pixels of the imagepickup device for example such as the exposure time of a main imagetaking are subtracted from the main image pickup signals at the time ofthe main image taking so that defects are detected with respect to thesubtracted image pickup signals of which dark components are offset. Itis thus possible to accurately detect the defects.

[0016] In accordance with a third aspect of the invention, the imagepickup apparatus according to the first aspect further includes atemperature detection means for detecting a temperature of the imagepickup device, and the control means controls the light blocking meansin accordance with the temperature detected at the temperature detectionmeans.

[0017] White-point defects of an image pickup device occur morefrequently as the temperature becomes higher. By detecting defects withrespect to subtracted image pickup signals resulting from thesubtraction of dark signals obtained by shutting out light for a timeduration corresponding to the temperature of the image pickup device asdescribed above, however, the defects can be accurately detected evenunder changing temperatures irrespective of mode setting such as of theexposure time.

[0018] In accordance with a fourth aspect of the invention, there isprovided an image pickup apparatus including: an image pickup device; adetection means for detecting fault pixels of the image pickup devicefrom image pickup signals of a selected frame by selecting apredetermined frame from image pickup signals consecutively outputted asa plurality of frames from the image pickup device; a storage means forstoring locations of the detected fault pixels; a correction means forcorrecting image pickup signals from the image pickup device on thebasis of the locations of the fault pixels stored at the storage means;and a control means for controlling timing at which the fault pixeldetection means detects fault pixels. Further, in accordance with afifth aspect of the invention, the fault pixel detection means in theimage pickup apparatus according to the fourth aspect detects faultpixels at every predetermined number of frames.

[0019] In the case of consecutively taking a plurality of frames, it isnot necessary to detect defects at each frame, since difference in imageis not large between consecutive frames. In the image pickup apparatusaccording to the above described fourth and fifth aspects, since defectsare detected by selecting a predetermined frame for example at everypredetermined number of frames from the image pickup signalsconsecutively outputted as a plurality of frames and the locations ofthe defects are stored to correct the defects, it is possible to correctthe defects without detecting defects at each frame. It is therebypossible to avoid easily cognizable fault pixels which occur due to thefact that the defects are seen and not seen by frames.

[0020] In accordance with a sixth aspect of the invention, the faultpixel detection means in the image pickup apparatus according to thefourth aspect detects fault pixels only at the first frame of a numberof consecutive frames.

[0021] When defects are to be detected by frame, the defect detectionprocessing takes time and performance of a further image taking withinsuch processing time may become impossible. The above problem, however,can be avoided by detecting defects only with respect to the first frameas described above.

[0022] In accordance with a seventh aspect of the invention, the faultpixel detection means in the image pickup apparatus according to thefourth aspect detects fault pixels at predetermined time intervals orduring an intermission of photographing after a main photographing.

[0023] An image taking at a desired point in time may become impossibleif time is consumed in defect detection processing. The above problem,however, can be eliminated by as described detecting defects atpredetermined time intervals or when image is not being taken.

[0024] In accordance with an eighth aspect of the invention, the imagepickup apparatus according to any one of the fourth to seventh aspectsfurther includes a motion detection means for detecting a motion inimage within a frame so that the fault pixel detection means detectsfault pixels with respect to a frame or a region within frame of which amotion quantity detected by the motion detection means exceeds apredetermined value.

[0025] Since, in consecutive taking of images, changes in image aresmaller between frames where motion of the object is small, it is notnecessary to detect defects at every frame. In the image pickupapparatus according to the above described aspect, since the motiondetection means is provided so that defects are detected when the motiondetecting quantity exceeds a predetermined value, it is possible not todetect defects at frames where motion is small.

[0026] In accordance with a ninth aspect of the invention, there isprovided an image pickup apparatus including: an image pickup device; astorage means for previously storing the locations of fault pixels ofthe image pickup device; an edge detection means for detecting edges inimage from image pickup signals outputted from the image pickup device;a fault pixel detection means for detecting fault pixels of the imagepickup device from a frame or a region in frame having a value of edgesless than a predetermined value on the basis of an output from the edgedetection means; and a defect correction means for obtaining locationsof fault pixels by adaptively switching between the fault pixeldetection means and the storage means based on an output from the edgedetection means so as to correct image pickup signals corresponding tothe obtained locations of the fault pixels. Here the storage meansincludes one that stores the locations of fault pixels at the time ofshipping from factory or one that stores the locations of fault pixelspreviously detected at the fault pixel detection means.

[0027] While errors tend to occur when defects in an image having manyedge components are to be detected, it is possible to accurately detectdefects by detecting the defects only with respect to those imageshaving relatively less edge components as described above.

[0028] In accordance with a tenth aspect of the invention, the imagepickup apparatus according to the ninth aspect further includes an edgereduction means for forming an image having reduced edges on the imagepickup device so that the fault pixel detection means detects faultpixels in a state where edges are reduced by the edge reduction means.

[0029] By such construction, defect detection at high accuracy can beperformed for example even in the case of taking an image of objecthaving many edge components.

[0030] In accordance with an eleventh aspect of the invention, there isprovided an image pickup apparatus including: an image pickup device; adetection means for detecting fault pixels of the image pickup devicefrom image pickup signals outputted from the image pickup device; astorage means for storing locations of the detected fault pixels; and adefect correction means for correcting image pickup signalscorresponding to the locations of the fault pixels stored at the storagemeans, wherein the fault pixel detection means detects fault pixelscorresponding to one frame by image pickup signals of a plurality offrames and stores the locations of the fault pixels to the storagemeans.

[0031] In the case where the processing speed becomes slower due to anincreased size of defect detection/correct ion processing, it isdifficult to perform and update the defect detection by each frame. Theabove problem, however, can be eliminated by changing regions in imageto be detected of defects by the frame as described so as to obtain adefect detecting result corresponding to one frame from several frames.

BRIEF DESCRIPTION OF THE DRAWINGS

[0032]FIG. 1 is a block diagram showing construction of a firstembodiment of the image pickup apparatus according to the invention.

[0033]FIG. 2 is a timing chart for explaining operation of the darksignal cancel section in the first embodiment shown in FIG. 1.

[0034]FIG. 3 shows the operation principle of the defect detectingsection in the first embodiment shown in FIG. 1.

[0035]FIG. 4 is a flowchart for explaining operation of the defectdetecting section and defect correcting section in the first embodimentshown in FIG. 1.

[0036]FIG. 5 is a block diagram showing construction of the defectdetecting section and defect correcting section in the first embodimentshown in FIG. 1.

[0037]FIG. 6 shows the relationship between exposure time to be set atthe exposure period controlling section and variable input parameters(threshold) in the first embodiment shown in FIG. 1.

[0038]FIG. 7 is a flowchart for explaining the total operation of thefirst embodiment shown in FIG. 1.

[0039]FIG. 8 is a block diagram showing a second embodiment of theinvention.

[0040]FIG. 9 is a block diagram showing a third embodiment of theinvention.

[0041]FIG. 10 is a timing chart showing an example of operation in thethird embodiment.

[0042]FIG. 11 is a flowchart for explaining an example of operation inthe third embodiment.

[0043]FIG. 12 is a timing chart for explaining operation of a fourthembodiment.

[0044]FIG. 13 is a timing chart for explaining an example of operationof a fifth embodiment.

[0045]FIG. 14 is a flowchart for explaining operation of the fifthembodiment.

[0046]FIG. 15 is a block diagram showing a sixth embodiment of theinvention.

[0047]FIG. 16 is a flowchart for explaining operation of the sixthembodiment.

[0048]FIG. 17 is a block diagram showing a seventh embodiment of theinvention.

[0049]FIG. 18 is a block diagram showing an eighth embodiment of theinvention.

[0050]FIG. 19 is a flowchart for explaining operation of the eighthembodiment.

DESCRIPTION OF THE PREFERRED EMBODIMENT

[0051] Some embodiments of the invention will now be described. FIG. 1is a block diagram showing a first embodiment of the image pickupapparatus according to the invention. The present invention is notlimited to black-and-white image pickup apparatus and can be applied toany type of image pickup apparatus including color image pickupapparatus. For ease of explanation, however, the present invention inthe embodiments below including the present embodiment will be shown asthat applied to an electronic camera which uses a black-and-white CCDimage pickup device.

[0052] Referring to FIG. 1, numerals are used to denote: 1, a lensthrough which an object light is caused to enter; 2, a light blockingplate; 3, a black-and-white CCD image pickup device forphotoelectrically converting the object light into electrical signals;4, an analog-to-digital converter for converting image pickup signalsoutputted from the CCD image pickup device 3 into digital signals; and5, an exposure period controlling section for controlling such as theexposure period of the CCD image pickup device 3 by controlling thelight blocking plate 2 and CCD image pickup device 3. Numeral 6 denotesa dark signal storing section consisting for example of DRAM for storingdark signals to be obtained from the CCD image pickup device 3 in astate where the incident light is shut out by the light blocking plate2; and numeral 7 denotes a subtracting section for subtracting darksignals stored at the dark signal storing section 6 from main imagepickup signals obtained from the CCD image pickup device 3 by a mainexposure where the light blocking plate 2 is retracted. A dark signalcanceling section is constituted by the dark signal storing section 6and the subtracting section 7.

[0053] Numeral 8 denotes a defect detecting section for detectingdefects from the subtracted image pickup signals obtained by subtractionof the dark signals at the dark signal canceling section; and numeral 9denotes a defect correcting section for performing correction of thefault pixels detected at the defect detecting section 8. It should benoted that the above described exposure period controlling section 5includes CPU or the like for managing the system. In addition to thecontrolling of the light shutting out timing at the light blocking plate2 and the charge accumulating time of CCD image pickup device 3 asdescribed above, it is adapted to adjust the dark signal storing timingat the dark signal storing section 6 and the defect detecting parameterat the defect detecting section 8.

[0054] Operation of thus constructed electronic camera will now bedescribed. First, a description will be given below with reference tothe timing chart shown in FIG. 2 of the operation until the productionof subtracted image pickup signal obtained by the subtraction of darksignal at the dark signal canceling section which is consisting of thedark signal storing section 6 and the subtracting section 7. At the timeof subtracting operation of dark signal, the light blocking plate 2 isfirst inserted into the optical path and, in the state where theincident light from lens 1 is shut out, an accumulation of dark chargecorresponding to the same time period as the exposure time of a desiredmain photographing is performed at the CCD image pickup device 3.Thereby a dark frame signal corresponding to the main exposure time fora desired photographing can be obtained from the CCD image pickup device3.

[0055] Next, the light blocking plate 2 is retracted from the opticalpath so as to start a main exposure image taking and at the same timethe dark signal is read out and the dark signal is stored to the darksignal storing section 6. Then, the main image pickup signal obtained bythe main exposure image taking is read out and at the same time the darksignal stored at the dark signal storing section 6 is read out. Thesubtraction processing of the two is performed at the subtractingsection 7. At this time, since dark components corresponding to the darksignal are also included in the main exposure image pickup signalobtained by the main exposure image taking, the subtracted image pickupsignal subtracted of the dark components is outputted by the subtractionprocessing at the subtracting section 7 as described.

[0056] For simplification in the above description of the dark componentsubtracting operation by way of the timing chart shown in FIG. 2, thedescription has been given of an example in one shot photographing usingCCD image pickup device 3 of an interline readout system where the framerate in readout is set to the same as the frame rate duringexposure/accumulation time and light is not shut out when readout isperformed.

[0057] Further, the above described embodiment has been shown as that inwhich the light blocking plate 2, dark signal storing section 6 andsubtracting section 7 are used to perform the subtraction of the darksignal of CCD image pickup device 3. It is however also possible thatthe main exposure image pickup signal is stored to the dark signalstoring section 6 so that a subtraction image pickup signal subtractedof dark signal is obtained by subtracting a dark signal similarlyobtained immediately after the main exposure image taking from the mainexposure image pickup signal which has been stored to the dark signalstoring section 6.

[0058] In the present embodiment, defects are detected at the defectdetecting section 8 and correction processing is performed at the defectcorrecting section 9 with respect to thus obtained output signal(subtraction image pickup signal) from the subtractor 7 which has beensubtracted of the dark signal. A description will now be given withrespect to the defect detecting operation and defect correctingoperation at the defect detecting section 8 and defect correctingsection 9. At the defect detecting section 8, the determination as todefect of observed pixel Xn is performed for example by an arithmetic asshown below so that defects are sequentially detected. Here “n” isdefined as n≧1 and represents the number of pixels in an inputted imagefile.

[0059] Specifically, as shown in FIG. 3, the observed pixel Xn and twopixels Xn−1, Xn+1 next thereto on the both sides are used so as todetermine whether it is faulty or not by the procedure indicated by theflowchart in FIG. 4. When determined as faulty, it is corrected. Inparticular, mean value A of the outputs of the observed pixel Xn to bejudged and of its two adjacent pixels Xn−1, Xn+1 is first obtained bythe following formula (1) (step S1).

A=(Xn−1+Xn+Xn+1)/3  (1)

[0060] The output of each pixel is then compared with mean value A so asto determine whether a necessary condition as indicated in the followingformula (2) is satisfied or not (step S2).

[A+b>Xn−1, A+b>Xn+1, A+b<Xn] or

[A−b<Xn−1, A−b<Xn+1, A−b>Xn]  (2)

[0061] where b is a defect detecting variable input parameter(threshold) having a value of b≧0, which is varied in accordance withthe exposure/accumulation time of the CCD image pickup device.

[0062] If the necessary condition indicated by the above formula (2) issatisfied, it is further determined whether a necessary condition of thefollowing formula (3) is satisfied or not (step S3).

|(Xn−1+Xn+1)/2−Xn|>a  (3)

[0063] where, like b, a is a defect detecting variable input parameter(threshold) having a value of a≧0, which is varied in accordance withthe exposure/accumulation time of the CCD image pickup device.

[0064] If the necessary condition indicated by the above formula (3) issatisfied, the observed pixel Xn is determined as a fault pixel. Thevalue of Xn determined as fault pixel is then interpolated by replacingit with (Xn−1+Xn+1)/2=B (step S4).

[0065] A description will now be given by way of FIG. 5 with respect toan example of hardware construction of the defect detecting section 8and defect correcting section 9 for performing the defect detectionshown in the flowchart of FIG. 4. Referring to FIG. 5, numeral 101denotes a pixel rearranging section for adjusting timings of the pixelsignals of the three pixels, i.e., the observed pixel Xn and the twoadjacent pixels Xn−1, Xn+1, including three flip-flops 11, 12, 13 eachfor causing one-pixel delay.

[0066] Numeral 102 denotes a defect detecting section, which includes: afirst adder 21 for obtaining a mean value by adding together the threepixel signals of Xn−1, Xn, Xn+1 (letting one obtained by the additioncorrespond to mean value A of the above formula (1), since the hardwareconstruction for division becomes large in size); multipliers 22-1,22-2, 22-3 for respectively tripling the three pixel signals; a firstcomparator 23 for comparing the value obtained by addition of the addedoutput A of the first adder 21 and variable input parameter b set andinputted from the exposure period controlling section 5 with therespective output of the multipliers 22-1, 22-2, 22-3; a second adder 24for adding the pixel signals of pixels Xn−1, Xn+1 which are adjacent tothe observed pixel Xn on the both sides thereof; an LSB cut circuit 25for acquiring ½ of the output of the adder 24 to obtain a mean; asubtractor 26 for subtracting the pixel signal of the observed pixel Xnfrom the output signal of the LSB cut circuit 25 (mean value B of thetwo pixels); a second comparator 27 for comparing the output of thesubtractor 26 with the variable input parameter “a” inputted from theexposure period controlling section 5; and an encoder 28 for determiningas to defect of the observed pixel Xn from the result of comparison atthe first and second comparators 23, 27. Here, the defect detectingsection 8 as shown in FIG. 1 is constructed by the pixel rearrangingsection 101 and the defect detecting section 102.

[0067] Numeral 103 denotes a defect correcting section, corresponding tothe defect correcting section 9 as shown in FIG. 1, which includes aselector 31 for selectively outputting the pixel signal of the observedpixel Xn or mean value B of the two adjacent pixels Xn−1, Xn+1 on thebasis of the determination output of the encoder 28 of the abovedescribed defect detecting section 102.

[0068] A description will now be given with respect to operation of thusconstructed defect detecting section 102 and defect correcting section103. First, at the pixel rearranging section 101: the pixel signal ofone of the adjacent pixels, Xn−1, is delayed by two pixels through thetwo flip-flops 12, 13 and inputted to the first adder 21; the pixelsignal of the observed pixel Xn is delayed by one pixel through the oneflip-flop 11 and also inputted to the first adder 21; and the pixelsignal of the other adjacent pixel Xn+1 is directly inputted to thefirst adder 21. At the first adder 21, then, these inputted pixelsignals Xn−1, Xn, Xn+1 are added together to compute a mean value (3A)corresponding to A of the formula (1).

[0069] On the other hand, the pixel signals Xn−1, Xn, Xn+1 to beinputted to the first adder 21 are also inputted to the multipliers22-1, 22-2, 22-3, respectively. The pixel signals of 3Xn−1, 3Xn, 3Xn+1outputted from the respective multipliers 22-1, 22-2, 22-3, the output3A of the first adder 21 and the variable input parameter b are theninputted to the first comparator 23 for comparison. In other words, itis determined at the first comparator 23 as to whether the conditionindicated in the formula (2) is satisfied or not, i.e., the conditionsof:

(A+b>Xn−1, A+b>Xn+1, A+b<Xn) or

(A−b<Xn−1, A−b<Xn+1, A−b>Xn).

[0070] Further, of the three pixel signals to be inputted to the firstadder 21, the adjacent pixel signals Xn−1, Xn+1 are inputted to thesecond adder 24 for addition. The added output is inputted to the LSBcut circuit 25 where it is reduced to ½ by cutting the least significantone bit (LSB) to obtain a mean value [B=(Xn−1+Xn+1)/2] of the adjacentpixel signals. At the subtractor 26, then, subtraction is performedbetween the mean value B of the adjacent two pixel signals and the pixelsignal of the observed pixel Xn. The result of the subtraction atsubtractor 26 is then compared with the variable input parameter “a” atthe second comparator 27 so as to determine whether the conditionindicated by the above described formula (3) is satisfied or not, i.e.,|(Xn−1+Xn+1)/2−Xn|>a. If both are satisfied in determining theconditions at the first comparator 23 and second comparator 27, theobserved pixel Xn is determined as faulty at the encoder 28.

[0071] In making determination as to whether the conditions aresatisfied at the first and second comparators 23, 27, a digital value“H” for example is respectively outputted if each inequality forindicating a condition is satisfied and “L” is outputted for othercases. The determination is made by forming all inequality outputsindicating the conditions into a logical value table, and being forexample coded by an encoder.

[0072] When the observed pixel Xn is determined as faulty at the encoder28, the defect correcting section 103 causes the selector 31 to outputthe mean value B of the adjacent pixels as a correction signal insteadof the pixel signal Xn of the observed pixel on the basis of the outputof the encoder 28. In other cases, the observed pixel Xn is determinedas a normal pixel and the pixel signal Xn of the observed pixel isoutputted as it is.

[0073] In solid-state image pickup devices such as CCD image pickupdevice, the amount of electric charge accumulated in a unit time greatlyvaries from one pixel to another including the effect of ambienttemperature. At the time of a long time exposure where charge iscontinuously accumulated for a longer period of time, the variance asdescribed becomes conspicuous as white point defects. The white pointdefects occur in every portion within the image. Such defects may beconcentrated in a certain portion within the image. The accuracy isgreatly reduced in detecting defects from the image in such case, sincethe defects are detected by using data of fault pixels themselves.

[0074] By contrast, in the present embodiment, it becomes possible toaccurately detect defects, since dark image to be subtracted is obtainedby continuously accumulating charge with shutting out light for the sameperiod as the image photographed by the main exposure and defects aredetected with respect to an image where such dark components arecanceled. Further, the variable input parameters (threshold) a, b foruse in detecting defects are varied in accordance with the chargeaccumulating time at the image pickup device so that it is made possibleto change the detecting accuracy according to the situation of thenumber of transiently occurring defects.

[0075] A description will now be given by way of FIG. 6 for indicatingthe relationship between exposure time and thresholds a, b and theflowchart of FIG. 7 with respect to the total operation of the presentembodiment with a detailed description of the setting of the variableinput parameters (threshold) a, b by the exposure period controllingsection 5. Since the level of white points is more conspicuous as theexposure time becomes longer, the variable input parameters (threshold)a, b are set as shown in FIG. 6 so that they become largerproportionally to the exposure time. T3 in exposure time of FIG. 6indicates the exposure time corresponding to the shortest exposure timeregion of those in the case of performing the dark signal subtractionprocessing. The values of thresholds a, b, i.e., the variable inputparameters at such exposure time are represented by a4, b4.

[0076] Upon start of photographing, it is first determined at theexposure period controlling section 5 whether the main exposure imagepickup time is longer than a predetermined exposure time Ts or not,based on which it is determined whether dark signal subtractionprocessing is to be performed or not. If, in the above determination ofexposure time, the exposure time is longer than the predeterminedexposure time Ts, a dark condition of the same time duration as the mainexposure time is produced by means of an operation of the light blockingplate 2. A dark signal obtained from the image pickup device 3 at thistime is taken into the storage section 6. The dark signal is subtractedfrom the main exposure image pickup signal to be taken immediatelythereafter. With respect to the subtracted image pickup signal obtainedby such subtraction, an identification is further made of the range inwhich the main exposure time is set. Particularly, in the exampleindicated by the flowchart of FIG. 7, it is categorized into the casesof: exposure time>T1; T1≧exposure time>T2; T2≧exposure time>T3; andT3≧exposure time>Ts. The thresholds a, b for detecting defects are thenoptimized according to the level of the categorized exposure time.Particularly, in the example indicated by the flowchart of FIG. 7,defects are detected by setting thresholds a, b to: a1, b1; a2, b2; a3,b3; a4, b4, respectively, and the defects are then corrected.

[0077] On the other hand, if the exposure period is shorter than Ts inthe determination as to whether the exposure time is longer than thepredetermined exposure time Ts or not, the dark signal cancelingoperation is not performed; and main exposure image pickup signals aretaken in immediately after the start of photographing so that defectdetection and defect correction are performed with respect to the imagepickup signals. At this time, after similarly identifying the range inwhich the main exposure time is set, defects are detected and correctedby optimizing thresholds a, b according to such level.

[0078] Particularly, in the example indicated by the flowchart of FIG.7, there are categories of: Ts≧exposure time>Ts1; Ts1≧exposure time>Ts2;Ts2≧exposure time>Ts3. For each of the categorized exposure time,defects are detected and corrected by setting the thresholds to: as1,bs1; as2, bs2; as3, bs3; respectively.

[0079] A second embodiment of the invention will now be described by wayof FIG. 8. The above described first embodiment has been shown as thatin which control of dark signal canceling operation and control ofthresholds a, b for detecting defects are performed in accordance withexposure time regarded as the external cause of defects, though it as aresult leads to an increase in temperature. In the second embodiment,irrespective of actual modes such as of exposure time, control of darksignal canceling operation and setting of thresholds a, b for detectingdefects are performed simply in accordance with the rise in temperatureof the image pickup device which is caused for example due to anexternal air temperature or operation time of the electronic camera.

[0080] In particular, as shown in FIG. 8, a temperature sensor 41 fordetecting temperature of the CCD image pickup device 3 is disposed inthe vicinity thereof. The exposure period controlling section 5 isadapted to control the dark signal canceling section and set thresholdsa, b for detecting defects in accordance with changes in temperaturedetected at the temperature sensor 41.

[0081] The relationship between the defect detecting thresholds a, b andthe temperature is similar to what is obtained by letting thetemperature correspond to the exposure time in the relationship betweenthe exposure time and thresholds as shown in FIG. 6. Further, the flowof camera operation is similar to an operation where exposure time isreplaced by temperature in the flowchart of the first embodiment shownin FIG. 7.

[0082] With an increase in the temperature, the charge excitation rateof the image pickup device is increased and the variance in chargeamount of each pixel becomes easily noticed, whereby white-point defectsare caused to occur similarly to the case of a long time exposure. Inthe present embodiment, however, defect detection/correction is renderedto a subtracted image pickup signal of which dark signal is canceled bythe dark signal canceling section so that correction can be performed byaccurately detecting defects. Further, since the variable inputparameters (threshold) a, b at the time of detecting defects are variedin accordance with changes in temperature, it becomes possible to changedetection accuracy according to the situation of the number oftransiently occurring defects so that erroneous detection can bereduced.

[0083] A third embodiment will now be described. In the case ofdetecting defects from an image, there is a possibility that faultpixels cannot be detected because of the effect such as of edge patternor noise in the image. In the case of consecutive photographing with alarge number of image pickup frames per second, motion of an objectbecomes slower as compared to the frame rate so that difference in takenimages becomes decreased between the consecutive frames. As long aschanges in the scene are not extensive as described, therefore, it isnot necessary to continuously detect defects at every frame and itsuffices to correct defects by means of once detected result of defects.

[0084] The third embodiment is adapted to deal with such conditions. Inparticular, as shown in FIG. 9, a defect location storing section 42consisting for example of DRAM for temporarily storing the result ofdefect detection is provided so that defects are corrected by using suchdefect location information until the stored contents have been updated.As the defect location information, it suffices for example to use pixelposition information in the horizontal and vertical directions withinthe image. It should be noted that this embodiment is applicable even tothe case of not providing a dark signal canceling section in the abovedescribed construction.

[0085] In thus constructed third embodiment, when images areconsecutively taken or when the camera is capable of varying frame rate,the exposure period controlling section 5 determines whether the framerate is higher than a predetermined rate or not i.e., whether frameperiod T is shorter than a predetermined period Th, and, based on this,controls the defect detecting section 8. In particular, the defectdetecting section 8 detects defects from a frame image immediately aftermaking such determination or at a timing close thereto, and the resultsthereof are stored to the defect location storing section 42.Thereafter, defects are detected and the defect location information isupdated at predetermined intervals after such frame. By thus performingdefect detection/correction, it becomes possible not to make the faultpixels easily cognizable due to their being seen and not seen frame byframe.

[0086] Of consecutively taken frames where one frame period is T, FIG.10 shows the case of 5 frames being a unit of defect detecting frameperiod Tk, i.e., the manner of performing defect detection by aninterval of 5T.

[0087] Further, FIG. 11 is a flowchart for explaining the operation ofthe present embodiment in the case of performing defectdetection/correction in the manner shown in FIG. 10. This operation maybe explained simply as follows. After the start of photographing,defects are detected anew if it is a consecutive photographing and ifframe period T is shorter than the predetermined period Th and if thedefect detecting period Tk has become longer than 5T. The defectlocation information is stored to the defect location storing section 42and defect correction is executed on the basis of such information. Inother cases, defect correction is executed by using the defect locationinformation which has already been stored to the defect location storingsection 42.

[0088] A fourth embodiment will now be described. The hardwareconstruction of this embodiment is similar to that of the thirdembodiment shown in FIG. 9. In this embodiment, as long as a high-speedconsecutive photographing is continued: defects are detected withrespect to the first one frame; and, after storing such defect locationinformation to the defect location storing section 42, defects arecorrected without updating the detection results of defect until thecompletion of such high-speed consecutive photographing. FIG. 12 showsthe manner of detecting defects only at the first one frame (indicatedby mesh) in the case where 12 frames are taken consecutively.

[0089] By such construction, for example in the case where time isrequired in the defect detection processing though it is desirable todetect defects by each frame, it is possible to avoid such problems asthat a frame cannot be taken afresh by taking a new photograph duringthe performance of defect detection.

[0090] A fifth embodiment will now be described. The hardwareconstruction of this embodiment, too, is similar to that of the thirdembodiment shown in FIG. 9. In this embodiment, the contents of thedefect location storing section 42 are updated: by detecting defects atevery predetermined time interval irrespective of the number of frames;or by detecting defects using time in a non-shooting condition wherephotograph is not taken for example when the electronic camera is in animage displaying mode, at the time of image storage processing or in animage editing mode.

[0091] When it takes time to detect defects as in the case of the fourthembodiment or when a defect detecting function is executed by softwareprocessing, there occurs a problem that the processing time of CPU isoccupied and it becomes impossible to take photograph. As in the fifthembodiment, however, the above problem can be avoided by detecting andupdating defects at every predetermined time interval or when photographis not being taken.

[0092] A description will now be given by way of the timing chart ofFIG. 13 with respect to a typical example in the present embodimentwhere, the technique is used of detecting and updating defects whenphotograph is not being taken. In the case where the electronic camerahas entered a non-shooting condition four frames after the start ofphotographing and photographing is to be resumed three framesthereafter, such non-shooting period, Tk1, is determined as a defectdetecting period. At this time, the defect detection is executed on thebasis of image information of the third frame which is the last of thetaken frames, and the defect location information is stored to thedefect location storing section 42. If photographing is resumed, thedefect detection is discontinued at that point in the processing and thephotographing is immediately started. If defect detecting operation withrespect to all the pixels within one frame is not complete in the firstdefect detecting period Tk1, the defect detection processing is executedin the next non-shooting period Tk2 for those pixels which have not beensubjected to the detecting operation and the defect location informationis stored to the defect location storing section 42. Further, it is alsopossible that a photograph corresponding to one frame (k1, k2) is takenonly for the purpose of defect detection during non-shooting to detectdefects on the basis of such frame k1, k2.

[0093] A description will now be given by way of the flowchart shown inFIG. 14 furthermore with respect to the technique for performing thedefect detection processing during non-shooting as described above. Itis first determined by the exposure period controlling section 5 whetheror not the electronic camera is in a shooting period where photograph isbeing taken. If not in a shooting period, the electronic camera entersthe defect detecting mode so that fault pixels in n pixels of the imagepickup device are detected and the fault pixel location information isstored to the defect location storing section 42. Such defectdetection/storage processing is performed for each one pixel so that, ifphotographing is resumed, the number of pixels n for which the defectdetection processing has been complete is stored to certain register andthe defect detection processing is temporarily discontinued. When anon-shooting period is resumed, the value of n is recognized so that thedefect detection operation is resumed starting from the pixel of whichdetection has not been complete. The above operation is repeated untilthe completion of defect detection with respect to all pixels k. Itshould be noted that correction processing is regularly executed for thephotographed frames on the basis of the defect location information.

[0094] A sixth embodiment will now be described. In the case of a framewithout a motion of the object in a consecutive photographing, it is notnecessary to detect defects for every frame, since changes in image donot occur. In the present embodiment, thus, a motion detecting section43 is provided as shown in FIG. 15 so that motion is detected at themotion detecting section 43 before the performance of defect detectionat the defect detecting section 8. The defect detection is performedonly with respect to those frames having a motion of the object or thoseregions in a frame image having a motion. For those frames without amotion of the object or those regions in a frame image without a motion,on the other hand, defects are corrected by using the defect locationinformation which has previously been obtained and stored to the defectlocation storing section 42.

[0095] As the technique for detecting motion at the motion detectingsection 43, it suffices to use for example a block correlation techniqueof two frames of image (such as the technique where an image is dividedinto a plurality of regions and the result of averaging the signallevels is computed by each block, thereby determination is made as towhether there is a motion or not depending on how the values at the samelocation are different from each other between the two frames). Itshould be noted that, since there is an image accumulating areacorresponding to one frame in the dark signal storing section 6 of thedark signal canceling section, it is also possible to use such storagesection to perform the frame difference computation in obtaining theabove described block correlation when the canceling of the dark signalis not performed.

[0096] The defect detection/correction operation in this embodiment willnow be described by way of FIG. 16. Only when value u resulting from anormalization of the motion detection quantity obtained at the motiondetecting section 43 is greater than a predetermined threshold value st,defects are detected and the defect location information is recorded atthe defect location storing section 42. In other cases, correction ofdefects is executed on the basis of the defect location informationwhich has already been obtained. The above described value, u, isobtained for example by a computation (difference) from the framecorrelation between the frame two frames before and the frame one framebefore the frame for use in the defect detection. Such value, u, can betreated either by frames or by blocks within one frame.

[0097] A seventh embodiment will now be described. The accuracy ofdefect detection largely depends on the manner of edges contained withinthe image. In the present embodiment, thus, an edge detecting section 44is provided as shown in FIG. 17 to detect edges at the edge detectingsection 44 before performing defect detection at the defect detectingsection 8. Defect detecting operation is then performed and the defectlocation information is updated only with respect to those frames havingless or low-level edges or those regions within a frame image havingless or low-level edges. As the technique for detecting edges by theedge detecting section 44, it suffices to use for example the techniquein which a high-pass filtering is effected so that determination is madeby verifying the components thereof.

[0098] While errors tend to occur in the case where defects are detectedwith respect to arbitrarily selected pixels within an image having manyedge components, the construction according to this embodiment iscapable of suppressing such effect so that defects can be accuratelydetected.

[0099] The operation of the seventh embodiment can be performed byequivalent steps as those in the flowchart for explaining operationrelated to the sixth embodiment shown in FIG. 16, by substituting aparameter obtained from normalization of a value representing the degreeof image edges for the value, u, which concerns the motion detectingquantity.

[0100] An eighth embodiment will now be described. In the abovedescribed seventh embodiment, the edge detecting section 44 is providedand edges are detected at the edge detecting section 44 before detectingdefects at the defect detecting section 8 so as to perform defectdetecting operation only with respect to those frames (regions) havingless or low-level edges. By contrast, in the present embodiment, animage having less edge components is deliberately produced on the cameraside, thereby making an accurate defect detection possible.

[0101] In particular, as shown in FIG. 18, a focus control and lensdrive section 45 is provided. It is provided with a defect detectingmode. When the electronic camera has entered the defect detecting mode,the focus control and lens drive section 45 is driven so that the lensis deliberately forced to be out of focus. Defects are then detectedwith respect to an image taken in that state and the result is stored tothe defect location storing section 42. Thereafter, the defects arecorrected by using such information.

[0102] While errors tend to occur in detecting defects of an imagehaving many edge components, the detection of defects is possible athigh accuracy even in the case of taking an object image having manyedge components by as described deliberately producing a defocused imagehaving less edges on the side of the camera to detect defects.

[0103] A flowchart for explaining the operation of this embodiment isshown in FIG. 19. The electronic camera enters the defect detectionprocessing mode and lens 1 is driven by the focus control and lens drivesection 45; and defects are detected when out of focus and the defectlocations are stored to the defect location storing section 42.

[0104] A ninth embodiment will now be described. If the size of defectdetection/correction processing is large and the processing speedthereof is slow, it is difficult to detect and update defects at everyframe. The present embodiment has been made to deal with such condition.While its hardware construction is similar to that of the thirdembodiment shown in FIG. 9, the image region for which defects are to bedetected is changed by each one frame so that a defect detecting resultcorresponding to one frame is obtained by several frames and is storedto the defect location storing section 42.

[0105] By such construction, it is possible to avoid the problems in thecase where it takes time in processing the defects and the problems suchas of eyesore due to the fact that defects are detected and not detectedfor example in taking a dynamic image.

[0106] As has been described by way of the above embodiments, it ispossible according to the first and second aspects of the invention toachieve an image pickup apparatus which is capable of accuratelydetecting and correcting defects even in cases where the defects occurin every portion of an image for example at the time of a long timeexposure. According to the third aspect of the invention, defects can beaccurately detected to perform correction even under the condition ofchanging temperatures. It is possible according to the fourth and fifthaspects of the invention to correct defects without detecting thedefects in every frame, since defects are detected by selectingpredetermined frames from the image pickup signals consecutivelyoutputted as a plurality of frames and the defects are corrected bystoring the locations of the defects. It is possible according to thesixth and seventh aspects of the invention to reduce the cases where anew image taking is impossible due to a defect detection processing.

[0107] According to the eighth aspect of the invention, it is possiblenot to detect defects with respect to frames where motion is small,since a motion detection means is provided so that defects are detectedwhen motion exceeds a predetermined value. It is possible according tothe ninth aspect of the invention to accurately detect defects, since anedge detection means is provided so that the defects are detected onlywith respect to those images having relatively less edge components.According to the tenth aspect of the invention, an edge reduction meansis provided so that defects can be accurately detected in a state whereedges are reduced even in the case of an object having many edgecomponents. According to the eleventh aspect of the invention, defectscan be detected and updated by each frame, since a defect detectingresult corresponding to one frame is obtained from several frames.

What is claimed is:
 1. An image pickup apparatus comprising: an imagepickup device; a light blocking means for shutting out light incidentupon the image pickup device; a subtraction means for subtracting a darksignal obtained as an output signal of the image pickup device whenshutting out the incident light from a main image pickup signal obtainedfrom the image pickup device at the time of a main image taking; acorrection means for correcting defect signals occurring due to faultpixels of said image pickup device with respect to the subtracted imagepickup signal from the subtraction means; and a control means forcontrolling said light blocking means consecutively before or after themain image taking so as to obtain a dark signal of a time durationcorresponding to the number of occurrence of externally caused faultpixels of said image pickup device.
 2. The image pickup apparatusaccording to claim 1, wherein said control means controls said lightblocking means in accordance with an exposure time of the main imagetaking.
 3. The image pickup apparatus according to claim 1, furthercomprising a temperature detection means for detecting a temperature ofsaid image pickup device, wherein said control means controls said lightblocking means in accordance with the temperature detected at saidtemperature detection means.
 4. An image pickup apparatus comprising: animage pickup device; a detection means for detecting fault pixels of theimage pickup device from image pickup signals of a selected frame byselecting a predetermined frame from image pickup signals consecutivelyoutputted as a plurality of frames from the image pickup device; astorage means for storing locations of the detected fault pixels; acorrection means for correcting image pickup signals from said imagepickup device on the basis of the locations of the fault pixels storedat the storage means; and a control means for controlling timing atwhich said fault pixel detection means detects fault pixels.
 5. Theimage pickup apparatus according to claim 4, wherein said fault pixeldetection means detects fault pixels at every predetermined number offrames.
 6. The image pickup apparatus according to claim 4, wherein saidfault pixel detection means detects fault pixels only at the first frameof a number of consecutive frames.
 7. The image pickup apparatusaccording to claim 4, wherein said fault pixel detection means detectsfault pixels at predetermined time intervals or during an intermissionof photographing after a main photographing.
 8. The image pickupapparatus according to claim 4 further comprising a motion detectionmeans for detecting a motion in image within a frame, wherein said faultpixel detection means detects fault pixels with respect to a frame or aregion within frame of which a motion quantity detected by said motiondetection means exceeds a predetermined value.
 9. An image pickupapparatus comprising: an image pickup device; a storage means forpreviously storing the locations of fault pixels of the image pickupdevice; an edge detection means for detecting edges in image from imagepickup signals outputted from said image pickup device; a fault pixeldetection means for detecting fault pixels of the image pickup devicefrom a frame or a region in frame having a value of edges less than apredetermined value on the basis of an output from the edge detectionmeans; and a defect correction means for obtaining locations of faultpixels by adaptively switching between said fault pixel detection meansand the storage means based on an output from said edge detection meansso as to correct image pickup signals corresponding to the obtainedlocations of the fault pixels.
 10. The image pickup apparatus accordingto claim 9 further comprising an edge reduction means for forming animage having reduced edges on said image pickup device, wherein saidfault pixel detection means detects fault pixels in a state where edgesare reduced by said edge reduction means.
 11. An image pickup apparatuscomprising: an image pickup device; a detection means for detectingfault pixels of the image pickup device from image pickup signalsoutputted from the image pickup device; a storage means for storinglocations of the detected fault pixels; and a defect correction meansfor correcting image pickup signals corresponding to the locations ofthe fault pixels stored at the storage means; wherein said fault pixeldetection means detects fault pixels corresponding to one frame by imagepickup signals of a plurality of frames and stores the locations of thefault pixels to said storage means.